During motion picture production, a picture and its associated sounds are typically recorded either on a common medium or on separate mediums. The picture is most often recorded on film, though recording on video is increasingly common. The associated sound is typically recorded on one or more audio tracks using either analog tape machines or digital recording devices. Because the sound is typically recorded on different media than the picture, resynchronization is required during the post production phase of filming.
To facilitate the resynchronization of the sound with the picture, a reference signal is often recorded along with the sound on the audio tracks. The reference signal may include a time code, typically of the SMPTFE format. The picture is apportioned into a number of frames, with each of the frames also including a SMPTE time code for indicating a relative position of the frame within the picture. Digital audio workstations, used during post production phase, utilize the SMPTE time codes of the motion picture and the SMPTE time codes of the audio tracks to control the speed of audio playback and thereby achieve resynchronization of the image with the audio.
Only some of the original frames of the motion picture will make it into the final production. The audio tracks, associated with those frames, may be retrieved using the SMPTE time codes. In addition, the audio track is evaluated for modification and improvement. The modification and improvement may be performed at a digital audio workstation (DAW). Digital audio workstations (DAWs), used to store and manipulate digital audio data, have gained popularity in the recording studio and post-production environments. A DAW is a computer that is used to modify and improve the audio track by adding tracks, building sound effects, and cleaning up existing tracks to remove unwanted background noise. Because the audio data is recorded in the digital medium, a number of advantages are provided over linear audio tape recording. For example, recording on a hard disk instead of a linear audio tape provides instant access to any portion of the audio track. In addition, the audio track may be edited and changed without affecting the original source material.
Once the audio editing is complete, the final film may be generated by playing back the audio tracks in synchronization with the SMPTE time codes of the selected picture frames. For top picture quality, the relationship between the display of the picture at a specified time code, and the production of the associated sound at that time code must be precisely synchronized. The loss of synchronization accuracy could potentially result in the degradation of the provided output image. For example, a gunshot must be heard at the same time a muzzle flash is seen. Of course, the synchronization accuracy requirements extend beyond Foley effects. The moving mouth of the speaker must be precisely synchronized with the dialogue being spoken, and a musical sound track for a feature film must be aligned with the unfolding action on the screen. In addition, synchronization may need to be maintained between multiple, different audio channels, regardless of any associated video.
In some video systems, a video slave driver such as the Digidesign Video Slave Driver.TM., is provided between the external video source device (video tape machine) and the digital audio workstation (DAW). The video slave driver is used to provide clocking and positional information to the DAW. The positional information allows the DAW to initiate playback of audio at a specified frame of the video. The clock provided to the DAW maintains synchronization between audio and video. During operation, the video slave driver is first used to synchronize the frame positions of the external device and the DAW. Once synchronization between the devices is established, the slave driver maintains synchronization, and, if it detects that synchronization has been lost, reestablishes synchronization between the two devices.
One method of the prior art to synchronize, or slave, a digital audio workstation to an external machine such as a video tape recorder includes the following steps. First, a clock reference is selected at the video slave driver, where the clock reference may be either a time code, such as the longitudinal time code LTC, a house video signal, or a video reference signal such as a black burst signal. A hardware clock resolver circuit uses the clock reference as the input to generate an audio sample clock. The audio sample clock is guaranteed to be locked to the clock reference with zero long term drift.
The audio sample clock and a positional reference (where both the sample clock and the reference may be the same signal) are forwarded to the DAW. Playback at the DAW is initiated through the use of a `trigger`, where the trigger may be, for example, the positional reference of the top of a video frame. The digital audio workstation is triggered to play back at a speed (1.times.) exactly in synchronization with the external device using the positional reference information that was input separately or provided as a clock reference. The digital audio workstation continues to play at 1.times. speed, with the clock resolver ensuring no long term drift between the digital audio workstation and the clock references. During playback, the positional reference received on the video signal is compared to the playback position of the digital audio workstations. When these two drift apart synchronization is lost.
One problem with synchronizing the digital audio workstation to the video tape deck is ensuring that the digital audio workstation is triggered to begin playback at a position aligned with the top of each video frame. For example, if the digital audio workstation were to trigger at the appropriate time code but not at the same time as the start of the frame, the audio playback may be unaligned with the video playback and the output effects would be degraded.
Typically, synchronization and triggering have been maintained by the use of software commands interfacing the slave driver to the digital audio workstation. During the synchronization steps outlined above, when the digital audio workstation triggers playback, it may not be perfectly aligned with the positional reference, either as a result of a fixed or a variable error. Fixed errors may include, for example, time code reader delays, transmission delays between the slave device and the digital audio workstation, playback engine startup delays at the digital audio workstation, the time for the digital audio workstation to detect a trigger event and audio interface transit delays. Variable delays may include those delays that occur as a result of software or as a result of hardware limitations. For example, a system interrupt may have incurred a servicing delay capable of affecting the relationship between the slave device and the digital audio workstation. Variable delays are problematic for users because systems response is not predictable.
To synchronize reliably, there are several methods a workstation may use to accurately trigger its playback engine at a particular external machine frame position. These methods depend on a time code reader's capability.
For example, one method of triggering is a poll and countdown method. In the poll and countdown method, a request for current time code information is sent by the DAW to the external device in the middle of a video frame. Once the time code of a particular frame has been determined, frame edges are counted to determine the exact beginning of a future time code for triggering purposes. Typically, the workstation will poll time codes in the countdown method until within a few frames of the trigger point and then count frame edges from there.
There are a few drawbacks to the poll and countdown method. First, this method is effective if the initial request for time code information is received at the external device in the middle of the desired time code. However, if the request is sent near a frame edge, it cannot be determined with precision whether or not the time code returned by the external device to the DAW is correct. Thus this method requires some way for the DAW to determine the middle of the frame for sending time code requests. Without such a mechanism, resultant triggering will be variable from up to one frame length. A further drawback of this method is that there are limits as to how rapidly the external device may be queried to determine the beginning, middle or end of the frame. Thus the accuracy of detecting the beginning of the video frame controls the trigger accuracy. Accordingly, polling delays and interrupt latencies may affect its accuracy.
A second method of triggering is a coincidence method. The VLAN and VLX video control LANS from Video Media support a coincidence code. The digital audio workstation instructs a time code reader at the video controllers to send a coincidence character when a particular time code is reached. The workstation uses the receipt of this character to trigger a playback.
One drawback of the coincidence method is that the accuracy of receiving the coincidence character determines the trigger accuracy. Thus, serial transmission time and serial interrupt latencies may affect the receipt of the coincidence character and thus may adversely affect the trigger accuracy.
Accordingly, a general aim is to provide a new method for triggering a playback engine to a particular external machine position.